Literature Review on Sparse Optimization

Some thoughts about algorithms for minimizing l and TV norms for signal and image recovery. In particular, I put the emphasis on the connections between Lagrangian and l constrained optimizations, and between analysis and synthesis priors. 1 Lagrangian and Constrained l1 Pursuits Lagrangian l pursuit reads ã = argmin b 1 2 ‖f − Φ∗b‖2 + T‖b‖1. (1) It is a Lagrangian formulation of the following constraint minimization ã = argmin b ‖b‖1 subject to ‖f − Φ∗b‖ 6 ε. (2) For each ε, there exists a value of T so that both problems (1) and (2) share a common solution [23, 35]. Iterative thresholding to solve (1). Several authors [12, 20, 2, 10, 36, 17] have proposed an iterative algorithm to solve (1). This algorithm iteratively performs a soft thresholding to decrease the l norm of the coefficients a, and a gradient descent to decrease the value of ‖f −Φ∗a‖2. This corresponds to the following iteration a[p] = ρT/τ ( a + 1 τ Φ(f − Φ∗a(k)) ) (3) where ρT is the soft thresholding operator ρT (α) = { 0 if |α| 6 T, α− sign(α)T otherwise. The iterates a converge to ã, the solution of (1) if τ > ‖Φ‖S , and in fact, [10] proves that it still converges if τ > ‖Φ‖S/2.